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AUTOMATED OPERATOR INSTRUCTION IN TEAM TACTICS 



ABSTRACT 



The 



Three questions are 
posed; Is there sufficient commonality in team tasks : p er f orm ed in existing 
team tactics trainers to warrant recommending deveiopment of a team ; . 
training system incorporating specific advanced technologies^ e. g, ; 
and adaptive techniques? If yes, what techniques should be used? If no, 
what other approaches are feasible for increasing the effectiveness of team 



Data were gathered on the team tasks 
training devices for air, surface 
data were analyzed for 



in representative 
These 
using 






Results indicated little commonality of team tasks; when tc^ap tasks 
inspected, Significantly more commonality was found when the stimulus, 
cognition and response elements of thev tasks were„,compared. A major 
recommendationisthattheNavyconcentraf eonimprovingtheeffectiveness 
of existing; devices rather ThanT^ a program jt^ ; 

training device. Furtherrecommendationsare made for thernianner in; 
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FOREWORD 



This report describes the results of a continuing effort of the Human 
Factors Laboratory at the Naval Training Device Center to identify and 
evaluate training techniques and principles for use in the design of devices 
which will enhance the training of Navy tactical teams. The principal goal 
of the effort reported herein was to determine the feasibility for application 
of advanced training technologies to the team training situation. Attention 
was directed towards determining the potentialities for generalized training 
and automated training principles to tactical team training. 

The analysis of team-task data from three team trainers representing 
air, surface and submarine operations indicated that the development of a 
generalized training system would not improve tactical team training. This 
approach was indicated to be appropriate for exercising only the routine 
and less-complex procedural tasks required of tactic * team members. For 
those tasks of greater significance in the tactical environment, the findings 
suggested that more efficient use and greater effectiveness of existing 
tactical team trainers could be realized through the adoption of part -team 
training for certain operator categories when coupled with the development 
and use of measurable behavioral objectives of training. There is a dis- 
cussion of selected principles and techniques which have been empirically 
demonstrated to offer increased training effectiveness. Also, several issues 
of theoretical and applied nature are discussed concerning the possible ap- 
plicability of these alternative techniques to tactical team training. This 
discussion is aimed at stimulating the necessary effort which could lead to 
a demonstration of the application of advanced training technologies in the 
tactical team setting. 




WILLIAM P. LANE 
Project Psychologist 
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SECTION I 
INTRODUCTION 



PROBLEM 

The Navy has procured many training devices with the objective of 
improving the communication skills and adaptability of tactical teams. 
Typically, such devices have been developed for each major weapon system. 
The general approach has been to develop training hardware which very 
closely resembles that used operationally. The assumption underlying this 
procurement philosophy is that high levels of task fidelity in the training 
environment are necessary for the adequate transfer of skills to the opera- 
tional world. 

The major objective of team tactics training is the improvement of team 
coordination. An assumption which must be met, however, is that each 
individual team member possesses the requisite individual skills for his 
position on the team. For example, before team training can realistically 
be conducted, sonarmen must know how to operate their sonar equipment, 
navigators their plot boards, etc. Yet, it has been found that team members 
often lack these prerequisite individual skills (Jeantheau, 1969; Schrenk, 
Daniels and Aid en, 1969). Consequently, team training devices are, of 
necessity, often used to develop individual rather than team skills. Although 
needed, such use compromises the major objectives of both the device and 
its intended use, i. e. , training team skills. 

Concern with the use, effectiveness, and cost of tactical team training 
systems has led to questions of whether advanced technologies such as 
generalised training, adaptive techniques, etc, , are feasible for providing 
the skills required of Navy tactical teams. For example, if the generalized 
team training concept is feasible, a single generalized training device could 
be used to train the skills required of team members regardless of the 
specific platform or weapon system they operate. This training device might 
produce significant increases in training effectiveness while, at the same 
time, reducing developmental costs. 

PURPOSE 

The purpose of this study was to determine the applicability of advanced 
technologies for team training. Specifically, the following questions were 



Is there sufficient commonality in the team tasks performed in 
existing team tactics trainers to warrant a recommendation 
for the development of a generalized team training system in- 
corporating specific advanced technologies? * 

If development is recommended, what techniques and tech- 
nologies should be incorporated into such a training system? 

If not, what other approaches are feasible to provide more 
effective tactical team training? 
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To place these questions in clearer perspective, a selective review of 
generalized and team training research was conducted. The following sub- 
sections summarize information concerning the value of generalized training 
and the manner in which team training should be conducted. 

REVIEW OF GENERALIZED TRAINING CONCEPTS 



Generalized training has been proposed as a means of reducing the costs 
associated with high-fidelity simulation while maintaining or improving 
training efficiency. To date, research on the concept has focused primarily 
on individual operator and maintenance skills. 



In the area of psychomotor behavior, such as fracking. Be wen. Hale, 
and Kelley (1962) developed and tested the concept of a General Vehicular 
Research Tool (GVRT), The feasibility of such a device, as noted by those 
authors, was based on the existence of a general tracking skill. The GVRT 
was proposed to provide the basic skill necessary for man to operate a 
dynamic closed-loop vehicle control sj'stem, yet not be specific, or neces- 
sarily sufficient, for any single vehicle. The GVRT was, therefore, pro- 
posed as a general trainer from which students would transfer to specific 
vehicles for final training once some minimum skill level had been attained. 

DePauli and Parker (1969), under sponsorship of the Naval Training 
Device Center (NAVTRADEVCEN), evaluated the feasibility of a general- 
purpose device for sonar maintenance training. This device, unlike the 
general trainer described above, is proposed for use at a Navy "C" school 
where basic knowledge of circuitry, electronics, components, etc. , is a 
prerequisite. Such maintenance training is normally performed on opera- 
tional equipment which is expensive, not designed for training purposes, fre- 
quently a generation behind that used in the fleet, and in insufficient supply. 
Consequently, such maintenance training has usually consisted of demonstra- 
tion, with little actual practice. The Generalized Sonar Maintenance Trainer 
(GSMT) was proposed as a solution to this problem. It incorporates all the 
basic circuitry and components found in operational sonar systems yet is 
organized in a fashion amenable to instruction on troubleshooting and cor- 
rective procedures. 

DePauli and Parker performed studies comparing performance on 
two types of maintenance trainers. One group was trained on the AN/SQS-4 
sonar (operational equipment, but no longer in fleet use) and the other group 
on the GSMT. Both groups were then transferred to the AN/SQS-23 sonar. 
The results indicated that GSMT trainees performed maintenance routines 
faster, made fewer procedural errors, and were as accurate as those trained 
on actual sonar equipment. An important additional finding was that training 
using the GSMT provided a means of equating end- of- training proficiency 
across differing trainee input aptitude levels, 

DePauli (1970) has suggested the feasibility and desirability of developing 
a generalized underwater fire-control system maintenance trainer. Lamb, 
Bertsche, and Carey (1970) recommended that the development of a general- 
ized submarine casualty control device for multiclass emergency ship control 
training is technically feasible and would be cost-effective. 
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The cognitive skills of problem solving and decision making require an 
ability to generalize from one situation to another. Sidorsky and his asso- 
ciates (Sidorsky, Houseman, and Ferguson, 1964; Sidorsky and Houseman, 
1966) studied tactical decision-making behavior with an aim of identifying 
trainable generalized decision-making skills. They concluded that similar 
deeisian-rnakinf? processes were involved in both ASW and AAW situations. 
Thev further observed that training was needed for situations where the 
decision-maker was at a tactical disadvantage. Training was observed by 
these authors to be directed toward training of specific procedures, when a 
different emphasis was required, i. e. , one stressing innovation and a dap - 
ability (Sidorsky, et al. , 1964). Hamm ell and Mara C 970) reported that 
despite differences in decision tasks there appeared to exist common decision- 
making skills. It was recommended that these skills be trained m the team 
training environment. 

Kanarick (1969), in reviewing the state of the art of decision-making 

research and training, suggested that decision making is a generaiizable 

skill Kanarick pointed out that, although decision making training has been 
wideiy adopted by industry, its acceptance by the military has been somewhat 
limited. Training In diagnosis and action selection could beneficially com 
element the procedural knowledge, doctrine, and system- specific- information 
braining Naval officers presently receive. Because such a Navy decision 
making course does not currently exist, Kanarick recommended that research 
be conducted to determine the best way to teach tactical decision-maxing 
skills. 

In summary, there exists a small body of literature which supports the 
feasibility of a generalized approach to training. However, all of the studies 
relate to individuals. Thus, although it can be shown that generalized training 
is effective for individuals in certain situations, the question remains as to 
whether this concept can be effectively applied to the training of teams. 

REVIEW OF TEAM TRAINING RESEARCH 

Training teams to effectively operate complex man-machine systems has 
been of concern to the military for the past i5 years. There is a significant, 
but unresolved, question of whether operation in a team setting requires 
unique skills that can only be developed and refined through team member 
interaction. Glanzer (1965), in discussing a series of studies of Navy team 
training and behavior, concluded that team training was inefficient and 
accordingly wasteful in terms of gain per trainee manhour. However, this 
same investigator stated that since the critical stimuli for individual tasks m 
the team context were difficult, if not impossible, to isolate, team training 
was necessary. 



Studies directed toward determining principles of team training have pro- 
duced conflicting results. For example, team-member replacement has been 
found in two cases to be relatively unimportant, with its effect at worst tem- 
porary and dependent on the skill level of the individuals involved (Horrocks, 
Krug and Heermann, 1 960 ; Briggs and Johnston, 1967), Yet, Schrenk 
Daniels, and Alden (i960), dealing with larger and more diffusely . structured 
tearks found turnover (member replacement) to be the major variable 
degrading team performance. Conflicting findings such, as these have made 
it difficult to determine optimum methods for team training. To resolve this 
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problem, complexes closely resembling the operational environment have 
typically been built for the training of real-world teams. In such devices 
"free play" tactics and terminal "kill or be killed" objectives have been 
emphasized (Jeantheau, 1969). However, questions remain regarding the 
effectiveness of rsuch devices and uses for significantly enhancing team per- 
formance. Therefore, research programs have been sponsored to address 
questions relating to team training and behavior such as: whether to train as 
individuals or in teams, the fidelity of simulation required, the optimal 
specificity of performance feedback, and the like. In this subsection, repre- 
sentative findings from three lines of research on team training (viz. System 
Development Corporation, American Institute for Research, and Ohio State 
University) are discussed with the objective of identifying results from which 
general principles of training technology might be derived, 

SYSTEM DEVELOPMENT CORPORATION (SDO). Alexander and Cooperband 
(1965), in a review of team behavior and research variables, described a 
team as a synthetic organism with individuals as components. The major goal 
of a te am training system was said to be development/ of system awareness 
within the team. This goal could be achieved through manipulation of the 
environment in an integrated fashion and through the provision of performance 
feedback information. A second goal of team training was to provide the 
skills for dealing with emergent, unstructured situations. Thus, training 
was depicted as a process of development from a task orientation, in which 
die environmental pattern must be recognized and the appropriate response 
procedure selected, to a "proceduralization" orientation, in which the teams 
applied creative responses to ambiguous situations. These investigators dis- 
cussed the criticality of variables associated with the task environment (work- 
load effects and knowledge of results) and task performance (practice 
schedules and degree of combination of individual and team practice) in the 
development of a team training technology. 

With regard to the specific issue of team versus individual training, 
Alexander and Cooperband proposed that whole-team practice should be effec- 
tive when: (a) the team training stresses the acquisition of coordinative 
skills, (b) the social facilitation provided has a beneficial effect on the acqui- 
sition of individual skills, (c) design of the tactical system is inadequate in 
that there is a discrepancy between the formal and informal rules of opera- 
tion, and (d), most Importantly, the tasks being trained are such that exhaus- 
tive formal rules cannot be stated and the procedures must be developed by 
the team in the process of task accomplishment. 

Although Alexander and Cooperband did not report empirical findings, 
the sections that follow contain data which bear directly on their suggestions. 

OHIO STATE UNIVERSITY. Horrocks and his associates performed a series 
of studies of team training for NAVTRADEVCEN (Horrocks, Krug and 
Heermann, I960; Horrocks, Heermann, and Krug, 1961). Using three-man 
teams in laboratory tasks such as jumbled-sentence decoding and target- 
position estimation, they found no increase in proficiency for individual skills 
resulting from a team practice situation. Rather, for the tasks used, indi- 
vidual competence was found to be a critical prerequisite for effective team 
performance. When team coordination was emphasized early in training, 
individual skill acquisition was hampered. Horrocks, et al. suggest that 
teaching individual skills in a team context may be inefficient. These 
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investigators recommended using part-task trainers with automated feedback 
devices for training individuals, an idea elaborated further in the Discussion 
section of this report. 



Other researchers at Ohio State University continued and expanded the 
NAVTRADEVCEN- funded investigation of team training. Briggs and Johnston 
(1967) have summarized four years of work on team training and have also 
Integrated these findings with those of other investigators. The results of 
this work will be discussed in some detail since it has important implications 
for the implementation of team training. 



It should be noted that the teams Briggs and Johnston studied were 
generally small (two operators and a supervisor), and some of their conclu- 
sions must be tempered by this fact. They recommended a hierarchal struc- 
ture for team organization. These authors pointed out that this organizational 
structure enables the hub, or decision-maker, of the team to elicit data 
exchange among team members, monitor the flow, and terminate communica- 
tion when the team objective is achieved or communication becomes excessive. 
Additionally, they reported that teams which used a minimum of interaction 
were more effective than more communicative teams. They interpreted tais 
finding in light of the fact that individuals have a limited channel capacity for 
information-processing. Communication requirements limit operator perfor- 
mance by occupying channel capacity which could otherwise be task-directed. 

Briggs and Johnston also suggested that parallel team structures are 
preferable to serial structures. This was proposed because team perfor- 
mance with the parallel structure is not dependent on the least skilled 
member. Also, the parallel arrangement permits workload adjustment 
between operators in redundantly manned team positions. However, Klaus 
and Glaser (1970), in reviewing the American Institute for Research (AIR) 
studies of team performance (discussed below), reported that the redundancy 
of the parallel structure led to only a short-term gain and eventually to a 
decrement in team performance. They suggested that the less skilled opera- 
tors will eventually interfere with their more skilled partners. 



Briggs and Johnston emphasized, not uniquely, the criticality of per for 
mance feedback in the acquisition of both individual and team skills. Feedback 
was found to be particularly important when the adequacy of an operator's 
performance was not easily determined from the task itself (e. g, , the task 
provided little intrinsic knowledge of results). A final conclusion reached by 
Briggs and Johnston concerned the appropriate mix of general and specific 
feedback for team training. They found that, if highly specific performance 
feedback were given too early in team training, it would actually interfere 
with skill acquisition. This was because trainees were not ready to use such 
specific information. The details of team performance should, according to 
these researchers, be provided only after a period of more general feedback. 
They also reported that teams tended to adjust their behaviors to maximize 
the incidence of specific feedback, even if such modified behaviors were 
inappropriate for attaining the team goal. 
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Bribes and Johnston concluded their review finding no direct evidence 
for the superiority of team training over Individual training. They inter- 
preted this finding as indicating a need for team training devices to include 
provisions for either refreshing or augmenting individual skills as well as 
providing team training* 

AMERICAN INSTITUTES FOR RESEARCH (AIR). The third and final line of 
research to be reviewed is the work done for the Navy by AIR. This research 
was summarized by Klaus and Glaser (1970). These authors characterized a 
team, as compared with a small group, as being relatively rigid m structure 
and organization and as possessing communication networks with well-defined 
member assignments and tasks. They suggested that a team can be treated 
as an intact entity and that total team performance can be modified through 
the type and amount of reinforcement provided. Yet, since the team output 
is a function of specific member inputs, one cannot deal with the team as a 
whole to the exclusion of the individual team members, 

Klaus and Glaser drew several general conclusions. They stressed that 
each correct response, whether team or individual, must be recognized m an 
explicit and prompt fashion. Performance monitoring and feedback, such as was 
observed by these researchers, however, was mainly oriented toward errors. 

Consistent with previous findings, the work of Klaus and Glaser indicated 
that the keystone of effective team training is individual proficiency. The 
team setting was neither an efficient nor appropriate place to acquire indi- 
vidual skills. Given that each team member knows his own taslc, the team 
setting is the place to learn communication and coordination skills. Klaus 
and Glaser’s data indicate that overall team performance levels pre- 
dieted from individual proficiency scores tend to he overestimates. This 
suggests that even with team members who are competent individuals, the 
necessity for interaction in a team setting subtracts from individual task 
performance. Thus, individuals must be trained to deal with this interaction 
and to function as team members. 

CONCLUSIONS. In summary, the literature cited above suggests the following 
conclusions: 

e Effective team training can only occur when each team member 
possesses a requisite level of skill. 

• Team training is a necessary adjunct to individual training, 

e A team organized in parallel with hierarchical control is the 
more effective team structure. 

• Performance feedback is critical to skill acquisition, A pro- 
gression from general to specific and from extrinsic to 
intrinsic feedback is best for skill acquisition. 
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Section ii 
method 



To determine the feasibility of a generalized approach to team training., 
it was necessary to identify the extent to which tasks were performed in com- 
mon within and among various team training environments. Additionally, 
information was obtained on current use of existing team training devices 
with the purpose of establishing how advanced technologies could be applied 
to increase training effectiveness. 

This section describes how and which trainers were selected for study, 
the framework used to describe operator tasks, the conditions of use of these 
trainers, and the techniques used to collect and analyze the data. 



TRAINER SELECTION 

Because this study allowed the investigation of only three tactical team 
trainers, one each for air, surface, and submarine operations, it was first 
necessary to ensure the representativeness of those selected. To aid in the 
selection process, definitions of the following terms were developed: 

• Team - Three or more persons working In concert toward 
a common, identifiable, and relatively immediate goal 

• Team Skill - Behavior which directly affects or influences 
some interface or relationship between individual members 
of a team 

• Tactics - The implementation of a cognitive plan or strategy; 
the key element is an action or inaction which attempts to 
optimize one's (vehicle) position within the environment or 
situation relative to an opponent or hazard 



Existing Navy team training devices were considered for inclusion in the 
study based on their representativeness and the relative cost of data collection. 

"Representative" was defined by a set of seven characteristics: 

• Designed to train team tactics 

• Designed to train three or more persons simultaneously 

• Had stations which are not located side by side (e, g. , as 
for a pilot- co- pilot trainer) 

• Included facilities for training maneuvering and/ or attack 

tactics (e. g. , engaging or tactically evading an enemy or 
other entity) 

• Represented a currently operational system with widespread 
use 
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e Controlled by a digital or hybrid computer 

® Had a special station from which instructor or operator per- 
sonnel can control problem parameters and monitor trainee 
p erf or mane e 

A review of existing Navy team tactic trainers identified a number of 
devices meeting the criteria of representativeness (see Appendix A), This 
list was rank-ordered in terms of relative representativeness and reviewed 
to determine the cost of including each device in this study. Emphasis was 
placed on selecting those for which access could be readily obtained and 
which had high levels of student use. Based on the above criteria and in con- 
sultation with NAVTRADEVCEN, a decision was made to study the following* 

@ Surface platform -- Emergency Ship Handling Trainer 

(Device 20A62) 

o Submarine platform -- Submarine ASW Attack Teacher 

(Device 21A38) 

• Aircraft platform -- P-3A Weapon System Trainer (Device 2F69B) 

Only Device 21A38 represented an optimal choice based on relative rep- 
resentativeness, Top-ranked devices for the air and surface environments 
were not available for study. Consequently, it was necessary to select the 
next most representative weapon system or platform control trainers. 

DESCRIPTION OF TRAINERS 

EMERGENCY SHIP HANDLING TRAINER (DEVICE 20A62). Device 20A62 
simulates the bridge environment for typical surface vessels. Each of its 
four bridge mockups can simulate the hydrodynamic characteristics of three 
classes of surface ships (DDG-2, LKA-112, and CVA-59). Device 20A62 pro- 
vides a capability for bridge teams to be trained in the areas of emergency 
shiphandling, maneuvering, in restricted and unrestricted waters, ship mal- 
functions, Rules of the Nautical Road, multiship maneuvers involving tactical 
formations, screens, search plans, etc, , maneuvering in fog, relative 
motion and maneuvering board problems, and communication procedures. 

Each bridge mockup provides stations for five active trainees: Officer of the 

Deck (OOD), Helm, Lee Helm-Engine Order Telegraph Operator (EOT), Radar 
Operator, and Horizontal Plotter, With in- team interactions occur through 
face-to-face verbal exchanges. The instructor may communicate with 
trainees from his separately located station. The major instructor-trainee 
communication channel is a general intercommunication network or simulated 
radio telephone (RT), Trainees in bridge mockup cubicles can communicate 
with one another thrpugh simulated RT nets and whistle signals. The major 
difference between the training and operational settings is the presence of a 
bird’s eye cathode ray tube (CRT) display in the training situation. This dis- 
play is computer- generated and depicts the positions and/or motion of the 
ship units and land masses involved in each problem. 
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SUBMARINE ASW ATTACK TEACHER (DEVICE 21A3 8), Device 21A3 8 pro- 
vides facilities for training submarine fire control and sensor system opera- 
tion. This trainer has facilities r simultaneously training three 16-man 
attack teams. Each separate attack teacher contains a simulated submarine 
attack center, sonar room, problem operator station, and a classroom. The 
control complexes provide facilities for problem direction, control, and 
observation. The three attack trainers can be used for independent team 
training or may be used to provide simultaneous training for three teams. 
Device 21A38 has three basic purposes: 

• Indoctrination and training of submarine fire control teams 
in a variety of situations 

• Formulation of new tactical doctrine and limited evaluation 
of new theories of submarine warfare strategy 

• A ccumulation and evaluation of performance data over 
numerous problem runs 

P-3A WEAPON SYSTEM TRAINER (DEVICE 2F69B). Device 2F69B is housed 
in two interconnected trailers, one containing the Tactics portion and the other 
the Operational Flight Trainer (OFT) portion. The two portions can be oper- 
ated individually in an uncoupled mode or simultaneously as a Weapon System 
Trainer (WST), The Tactics portion provides a very-high-fidelity simulation 
of the systems for each of the five weapon system team members: Radar/ 

MAD operator, Julie/ ECM operator. Navigator (NAV), Tactical Coordinator 
(TACCO) and Jezebel/AQA-1 operator. The device uses government-furnished 
equipment (GFE) at each trainee station, thus providing exact faceplate and 
control simulation. Displays and functions of the various GFE equipment are 
controlled by a central computer. A 30- inch-square optical plotter displays 
aircraft, sonobuoy, weapon, and target events, A separate instructor/ 
operator station is located immediately behind the trainee stations, providing 
instructors with direct (via a window) and indirect (via indicators) trainee 
monitoring capability. The instructor's station consoles provide controls, 
displays, and status indicators for monitoring and controlling the training 
situation. 

Team positions for each of these three devices are shown in Appendix B. 
DATA COLLECTION 

Data was collected by a team of four Honeywell behavioral scientists 
(hereafter referred to as the Honeywell Study Team, HST). The data collec- 
tion process involved four steps: liaison, staff briefings, data collection, and 
staff debriefings. Details of liaison, briefings, etc, are discussed in 
Appendix C. 

The HST manually recorded verbal representations of the team tasks per- 
formed by all team members at each device. Later appropriate taxonomic 
classification was assigned to each task statement. Because these activities 
were based on a specific definition of a task and on a taxonomic structure for 
handling task descriptions, the following sections will discuss that definition 
and taxonomy. 
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TASK TAXONOMY 

To organize team task information, a task classification scheme was 
used which permitted description of tasks in terms of a numerical code. A 
modification of Yaeger's taxonomy (1969) was used because it offered a rela- 
tively precise way to describe tasks. Other taxonomies considered (as 
reviewed by Ginsberg, McCullers, Maryman, Thomson and Witte, 1966) 
were too general for the purposes of meaningfully determining task common- 
ality. A taxonomy was needed which would allow removal of the situation 
specific task aspects but which would indicate the general stimulus, cognition, 
and response characteristics of each task. 

The structure of Yaeger’s taxonomy centers on the three elements 
reflected in his definition of a task: stimulus (S), cognition (C), and response 

(R), Once these elements are identified for a task, each is transformed into 
a numerical representation reflecting the specificity of description being used. 
For this study, a seven-position numerical code was used to describe each 
task. Of the seven positions, the first two described the Stimulus Modality 
(1) and Stimulus Information Uncertainty (2), The next three positions were 
used for the Cognition categories of Perception (3), Information Processing 
(4), and Action Selection (5). The final two positions described Response 
Modality (6) and Response Complexity (7). Thus, all tasks for this study are 
described using a seven-position code with the characteristics shown in 
Table 1. 

Categories, in turn, contain levels; e, g. , the levels of the Stimulus 
Modality category, include visual, aural, tactual, etc. Each category 
(Modality, Information Uncertainty, etc. ) is given a position in the numerical 
code description. Table 1 also shows the verbal description of a typical task 
along with its coded numerical representation. 

The digits assigned to each position in the example code reflect the 
specific level of the appropriate S, C, R category. Digits in the example 
code shown in Table 1 show that the stimulus was received aurally 
(2X -XXX -XX) and contained one bit of certain information (X1.-XXX-XX). 

The levels of cognitive elements for the example task indicate that the stimu- 
lus was recognized (XX-3XX-XX), and the data was analyzed for appropriate 
action (XX-X1X-XX) which was following a specified procedure (XX-XX2yXX), 
The resulting response was motor (XX-XX-3X) and quite simple (XX-XX-X1J. 
Thus, the numerical code of 21-312-31 describes this example task. 

TASK DESCRIPTION DATA 

Verbal descriptions of team tasks were obtained for all team members in 
each device studied. Members of the HST observed the activities of the same 
single team member during the conduct of a training exercise (at least two 
HST members visited each device). They first recorded verbal descriptions 
of the response, then the stimulus, and finally inferred cognitive activity. 
When questions existed regarding the stimulus or cognitive aspects of a given 
task, this matter was reviewed with the appropriate trainee. Because data 
collection at each device often bridged a number of training exercises and dif- 
ferent teams, the final listing of task description statements reflected a single 
list which was obtained for each trainee station, but not necessarily for only 
a single trainee in that position. 



TABLE 1. TAXONOMIC STRUCTURE AND NUMERICAL CODING SCHEME 
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One problem encountered was that of collecting task- sequence data for the 
various team positions. Because the lists of verbal descriptions were often 
formulated from observing a number of operators in a single position, and 
since several different problem exercises were usually observed across oper- 
ators, task-sequence information could not be reflected in the task description 
lists. 

CONSOLIDATION OP INDIVIDUAL TASK DESCRIPTION LISTS 

Once members of the HST had observed all operator positions and had 
formulated their respective individual task description lists, a single "master" 
list was prepared for each trainer. This master list resulted from consolida- 
tion of individual HST lists. 

During the consolidation process, tasks were considered one at a time, 
affording an opportunity for the HST to discuss and ensure the accuracy of 
each task description. An attempt was made to eliminate duplication or 
repetition of tasks within the listing for each operator position. This process 
resulted in a single, exhaustive list of task descriptions for each position 
within a given trainer. This list of task statements was then reviewed with 
the trainer staff for completeness and accuracy. In the situations where a 
staff member had suggestions for modifying the list, further observation of 
the training situation was made to obtain information which could resolve the 
discrepancies. 

DATA ANALYSIS 

CODING OF TASK DESCRIPTIONS INTO THE NUMERICAL TAXONOMY. 

When a master list of verbal task descriptions was completed for each team 
position in each of the three trainers, numerical codes were assigned to the 
verbal task statements according to the taxonomy (see Appendix D), The same 
HST personnel who developed the verbal task statements assigned these nu- 
merical codes. Additionally, a coding system was devised for identifying each 
task according to the person who performed the task. This scheme was used 
to permit exact identification of tasks for later analysis. Based on the major 
task functions performed, trainees were assigned to one of five Operator 
Categories. The Operator Categories and assignments by device are shown 
in Table 2. 

DATA PROCESSING. The coded data was analyzed to determine the common- 
ality of tasks among the three training devices and among various categories 
of team personnel. The task commonality between any pair of devices was 
defined as the number of exact correspondences of numerical task codes for 
tasks accomplished in those devices, A commonality index was obtained for 
five different task code digit sequences. In addition to determining the num- 
ber of total or seven-digit task codes common to the various devices, four 
other analyses were run using only selected elements of each seven- digit code. 
A shaded area is shown in Table 3 for those categories included in these five 
analyses. Thus, the second analysis shown in Table 3 indicates that only the 
Stimulus Information Uncertainty, Cognitive Information Processing, and 
Response Complexity task categories were required to match for commonality 
to exist. Other partial code analyses were run for stimulus elements only, 
cognitive elements only, and response elements only. Each of the five partial- 
code analyses produced a different number of unique task codes. This was 
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TABLE 2. OPERATOR CATEGORY ASSIGNMENTS BY DEVICE 





Device 


Category 


2 0A62 


21A38 


2F69B 


1 

Decision * 
Making 


• Qfiicer of the Deck 
(OOD) 


0 Conning Officer 
(CONN) 

o B ire Control 

Coordinator (FCC) 
o Sonar Supervisor 


• Tactical Coordinator 
(TACCO) 


2 

Plotting 


• Horizontal Plotter 


• Plot Coordinator 

0 Time-Bearing Plotter 

• Strip Plotter 

• Lynch Plotter 


• Navigator 


3 

Sensor 

Operation 


• Radar Operator 


# BQR-2 Operator 
0 BQR-7 Operator 


• Jezebel Operator 
0 Julie Operator 

• Radar Operator 


4 

Continuous 

Tracking 


0 Helm 


• Attack Director 
Operator (ADO) 

• Analyzer Operator 
(AO) 

• Weapon Control 
Console Operator 
(WCCO) 




5 

Discrete 

Tasks 


0 Lee Helm 


0 Expanded Time- 
Bearing Plotter 
• Time-Bearing 
Talker 

0 Time-Bearing 
Recorder 


-i 



TABLE 3. TASK CODE ELEMENT COMBINATIONS ANALYZED 
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true because fewer digits were required to match than in the case of the seven- 
digit analysis, and thus fewer digit combinations were possible. For a given 
task code sequence analysis (e. g. , code position X2-X4X-X7) each task was 
identified as being! (a) performed within a single device; (b) performed within 
two devices; or (c) performed within all three devices. Similar analyses were 
made for Operator Categories, i. e. , determining which tasks were performed 
by members of one, two, three, four, or five Operator Categories. A tabula- 
tion was also made specifying the devices and Operator Categories associated 
with each specific task code, 

NONTASK DATA 

Data were also collected to identify areas where new or different training 
technology could be applied to increase the effectiveness of current team 
training. These data represent a profile of practices currently used of the 
study devices, and include information pertaining to the following general 
areas: 

• Training objectives 

9 Trainee evaluations 

• Training methods, types, and materials 

m Instructor personnel 

• Trainee personnel 

• Instructor-trainee interaction 
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SECTION m 
RESULTS 



TASK CODES 

When codes were assigned to the 509 unique verbal task descriptions, 
289 different seven-digit codes resulted. Based on the number of taxonomic 
levels used, 69, 120 combinations of the seven digits were theoretically pos- 
sible. Figure 1 shows the number of task descriptions and associated codes 
■which were unique and common to the various devices. Only two codes, 
14-212-22 and 23-312-23 were common to all three devices. These two 
codes accounted for eight of the 509 task descriptions. Thus, at this level 
of analysis, three-way or total device commonality was only 1. 6 percent. 




12-345-67 CODE 

NO. OF TASKS 
(NO. OF CODES) 



Figure 1. Task Frequency and Task Code Uniqueness and 
Commonality for Seven-Digit Code Analysis 
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COMMONALITY BY DEVICE 

Little task commonality was found between Device 20A62 and Devices 
21A38 and 2F69B (5.7 and 3.2 percent, respectively). However, common- 
ality between Devices 21A38 and 2F69B accounted for 69 task descriptions 
(or 15. 6 -percent overlap). Approximately half of these descriptions (29) 
were coded as 12-3 12 -34 or 22-312-34, differing only in Stimulus Modality. 
Beyond this difference, the character of codes common to Devices 21A38 
and 2F69B may be depicted as involving discrete, relatively certain stimuli, 
procedure -following cognitive actions and a complex, skilled, continuous 
motor response. 

Codes found to be unique to a single device contained a wide range of 
characteristics. Additionally, a single seven-digit code seldom accounted 
for more than one or two task descriptions. Those tasks appear less singu- 
lar, however, when the specific code levels which describe the nature of 
various tasks were inspected more closely. 

Table 4 shows the frequency with which the various S-C-R task element 
levels occurred in the 509 tasks. Based on these frequencies certain unique 
combinations of taxonomic levels were associated with specific study devices. 
For example, in Device 21A38 a stimulus modality level of 2 accounted for 
141 of the 231 tasks (or 61 percent) accomplished in that device, while in 
Device 2F69B stimulus modality level 1 characterized 57 percent of its tasks. 
By using the most frequently occurring code element, the study devices may 
be characterized by the following S-C-R composites: 

20A62: 13-212-23 or 13-312-23 

21A38* 22-312-22 

2F69B: 12-312-34 

None of these composite codes contain code levels which are difficult or " 
complex in and of themselves. 

Four additional analyses were conducted to define more clearly the 
commonality of tasks. These analyses were concerned with specific parts 
of the seven-digit task codes. The partial analyses performed were: Com- 
plexity Index (Figure 2), Stimulus Elements (Figure 3), Cognitive Elements 
(Figure 4) and Response Elements (Figure 5). In these figures the number 
shown in the parentheses, indicate the number of task codes, while the num- 
bers which stand alone show task descriptions accounted for by such codes. 

COMPLEXITY INDEX ANALYSIS. For the Complexity Index analysis, three 
S-C-R elements from the taxonomy were used; Stimulus Information Uncer- 
tainty, Cognition Information Processing, and Response Complexity. As 
Figure 2 shows, a large number (254 of 509) of the task statements were 
found to be common to all three devices. The most frequently occurring 
Complexity Index codes were: X2-X1X-X2 (N=54), X2-X2X-X2 (N=25), 
X4-X1X-X2 (N=25), X3-X1X-X3 (N=22). These codes indicate some variety 
in the complexity of stimulus and response elements, but with a preponderance 
of discrete single -parameter or level 2 codes. The Cognitive Information 
Processing level, on the other hand, contained only two levels. Data Analysis 
arid Problem Diagnosis. 
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TABLE 4, FREQUENCY OF TAXONOMIC ELEMENT LEVEL 
BY DEVICE 



•— — ■ — •• •• 

Element Level 


Device 




20A62 


21A38 


2F69B 


Total 


Stimulus 


0 


0 


0 


0 


0 


Modality 


1 


34 


48 


121 


203 




2 


29 


141 


52 


222 




3 


0 


0 


0 


0 




4 


0 


35 


26 


61 




5 


4 


7 


12 


23 


Stimulus 


0 


0 


O 


0 


0 


Inf or mat ion 


1 


4 


17 


11 


32 


Uncertainty 


2 


13 


139 


77 


229 




3 


22 


31 


41 


94 




4 


10 


26 


45 


81 




5 


14 


10 


29 


53 




6 


4 


8 


8 


20 


Cognitive 


0 


9 


4 


0 


13 


Perception 


1 


2 


16 


24 


42 




2 


18 


42 


12 


! 72 




3 


18 


146 


121 


285 




4 


13 


13 


48 


74 




5 


7 


10 


6 


23 


Cognitive 


0 


1 


52 


0 


53 


Information 


1 


26 


89 


121 


236 


Processing 


2 


15 


49 


56 


120 




3 


25 


40 


34 


99 




4 


0 


1 


0 


1 


” -- ILL- U , 

Cognitive 


0 


6 


4 


0 


10 


Action 


1 


11 


25 


31 


67 


Selector* 


2 


39 


192 


148 


379 




3 


11 


10 


32 


53 


Response 


0 


0 


0 


0 


0 


Modality 


1 


3 


5 


24 


32 




2 


48 


155 


55 


258 




3 


12 


65 


110 


187 




4 


, 4 


6 


22 


32 




5 


0 


0 


0 


0 


Response 


0 


3 


5 


0 


8 


Complexity 


1 


7 


17 


37 


61 




2 


21 


130 


60 


211 




3 


27 


4 1 


41 


109 




4 


4 


34 


61 


99 




5 


S 


4 


12 


21 




6 


0 


0 


0 


0 
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X2-X4X-X7 CODE 



NO. OF TASKS 
(NO. OF CODES) 



Figure 2. Task Frequency and Task Code Uniqueness and 
Commonality for Three-Digit Complexity 
Analysis 



12-XXX-XX CODE 



NO. OF TASKS 
(NO. OF CODES) 



Figure 3. Task Frequency and Task Code Uniqueness and 
Commonality for Two-Digit Stimulus Analysis 
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XX-345 XX CODE 



NO. OF TASKS 
(NO. OF CODES) 



Figure 4. Task Frequency and Task Code Uniqueness and 
Commonality for Three-Digit Cognition 
Analysis 




XX-XXX-67 CODE 



NO. OF TASKS 
(NO. OF CODES) 



Figure 5. Task Frequency and Task Code Uniqueness and 
Commonality for Two-Digit Response Analysis 
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Complexity Index codes which were common to two devices revealed a 
situation similar to that found in the two-device overlap of seven-digit codes. 
That is a much greater similarity between Devices 21A38 and 2F69B than 
between either of these devices and the 2OA02 (i. e, , 24. 9 percent overall 
versus 4. 7 and 5, 8, respectively). The Complexity codes found to be com- 
mon between Devices 21A38 and 2F69B may be characterized by stimulus 
and cognition of fairly low complexity. Responses, however, covered a 
wider range of levels and were generally of higher complexity. 

The Complexity Index codes unique to single devices accounted for only 
23 percent of the task statements. This was true because each code normally 
represented only one or two task statements. For example, eight Complexity 
Index codes were unique to Device 20A62. Those codes represented eight dif- 
ferent task descriptions of wide ranging complexity. 

Of the task statements occurring uniquely in Device 21A38, 29 could be 
coded using only two codes. Those codes were X2-X0X-X2 and X2-X0X-X4, 
Thus, minimal stimulus and cognitive complexity but somewhat more response 
complexity characterized approximately half of the tasks observed uniquely m 
Device 21A38, 

The characteristic complexity of tasks accomplished uniquely in Device 
2F69B was found to be generally higher than for any other device. However, 
there was no single Complexity Index code which occurred significantly more 
frequently than any other. 

STIMULUS ELEMENT ANALYSIS. The partial code analysis of stimulus 
elements also indicated large commonality among devices. In this analysis, 

11 task codes accounted for 423 of the total 509 task statements (see Figure 3). 
Of the Stimulus codes which were common to all three study devices, a single 
code ( 2 2 -XXX -XX ) indicating a low-complexity auditory stimulus characterized 
138 different task statements. The second most frequently occurring code was 
12-XXX-XX indicating a visual as opposed to aural stimulus modality. The 
stimulus elements common to all three devices typically involved either visual 
or aural Stimulus Modality but covered the complete range of Information 
Uncertainty. 

Stimulus elements found to be common to two devices occurred only be- 
tween Devices 20A62 and 2F69B (2. 5 percent) and between Device 21A38 and 
2F69B (17, 2 percent). The codes found in each of these common areas usually 
involved a combination aural- visual Stimulus IVIodality. The Information Un 
certainty portion of these stimulus elements, however, covered the total pos- 
sible range of case levels (1 to 6). 

COGNITIVE ELEMENT ANALYSIS. Analysis of the cognitive elements alone 
revealed that the majority of the codes were again common to all three devices 
( see Figure 4). Of the 10 codes common to the three devices, a single code 
(XX-312-XX) accounted for 159 of the 298 task statements. Thus, most of 
the tasks observed in the study devices required procedure -following- type 
cognitions (-312-). When the minor variations of this cognitive combination, 
e.g. , -212-, -302- and -4 12-, are included in the tally, nearly half (45.6 
percent) of the total number of descriptions were accounted for. 
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A similar result was obtained from inspection of the cognitions char 
acterizing commonality between only two devices. Considerable variability 
was noted in the Perceptual and Action Selection cognitive elements, with 
the Information Processing category nearly always coded at level 1 or 2 
(data analysis or problem diagnosis). 

Cognitions unique to single devices typically included higher -level (i. e. , 
Identification, Classification) perceptions. The Level 0 Information Process- 
ing code occurs here for the first time in Devices 20A62 and 21A38. A Level 
0 Information Processing code indicates a task involving little or no process- 
ing of stimulus information, such as reflexive obedience to an order. Again, 
the Action Selection portion of these cognitions mainly involved the decision 
to follow a specified formal procedure, 

RESPONSE ELEMENT ANALYSIS. In the analysis of response elements, a 
vast majority of task statements (86. 6 percent) were found to be common to 
all three devices (see Figure 5). Codes representing the greatest number of 
task ltotemlntf were XX -XXX - 2 2 <N= 168) XX -XXX -34 <N=87). and XX -XXX- 
23 (N=65). Verbal responses accounted for 58, 5 percent and motor responses 
36, 3 percent, and the remaining 23 tasks involved a combined response mod- 
ality. The complexity of these response codes ranged from Levels 1 to 5 with 
the highest frequency (45 percent) at Level 2, which is single -parameter, dis- 
crete. 

Few response element codes were found to be common to only two devices 
or unique to a single device. Of those unique to a single device, the responses 
for tasks accomplished in Device 2F69B were often "visual orienting m nature. 
This finding reflects the fact that Device 2F69B contains many redundant dis- 
plays and that team members relied heavily on such displays for interacting 
with one another, 

COMMONALITY BY OPERATOR CATEGORY 

Table 5 shows the number of task statements and associated codes for 
each of the five analyses by operator category. 

Inspection of the data in Table 5 indicates a level of task commonality 
among operator categories similar to that found when the data were analyzed 
on a device basis. For example, relatively few tasks (23 of 509) were found 
to be common to all five operator categories with the seven-digit task code 
analysis. Rather, a majority of the task statements and seven-digit codes 
were unique to single operator categories. Again, this finding parallels t e 
results obtained in the device commonality analysis. When the specific num- 
ber of codes unique to single categories were inspected, it was found that per- 
sonnel in Category 1 (decision makers) and in Category 3 (sensor operators) 
perform the largest number of unique tasks: 133 and 131 respectively. 

However, when the partial code analyses were done, a vast majority of 
the 509 task codes were common to more than one of the five operator cate- 
gories. For the Complexity Index analysis, nearly all tasks were accomplished 
in common by the operators in two or more categories. Analyses of the h, G, 
and R elements alone showed that 499, 483, and 485 task descriptions, respec- 
tively, were done in common by persons in two or more categories. 
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TABLE 5 NUMBER OF TASK CODES AND DESCRIPTIONS UNIQUE 
TO AND IN COMMON AMONG OPERATOR CATEGORIES 



— - — 


Task Codes 




Operator 


I 2-34 5-0 7 


X 2 -X 3X 


-X7 


12 -XXX 


-XX 


XX -34 5 


-XX 


xx -xxi 


X -67 


Category 51 ' 


Codes 


Tasks 


Codes 


Tasks 


Codes 


Tasks 


Codes I 


Tasks 


Codes 


Tasks 


1 


104 


133 


17 


29 


1 


2 


12 


19 


1 


2 


2 


31 


45 


3 


4 


1 


1 


2 


2 


0 


0 


3 


97 


131 


21 


43 


1 


7 


3 


5 


4 


22 


4 


20 


22 


1 


2 


0 


0 


0 


0 


0 


0 


5 


3 


4 


0 


0 


0 


0 


0 


0 


0 


0 


i+2 


6 


11 


2 


9 


1 


3 


3 


14 


0 


0 


1 + 3 


4 


15 


14 


64 


5 


45 


8 


49 


2 


15 


1+4 


2 


7 


2 


5 


0 


0 


2 


9 


0 


0 


1+5 


1 


3 


0 


0 


0 


0 


0 


0 


0 


0 


2+3 


3 


15 


1 


3 


0 


0 


0 


0 


1 


4 


2+4 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


2+5 


1 


2 


1 


2 


0 


0 


0 


0 


0 


0 


3+4 


5 


23 


3 


18 


0 


0 


1 


11 


0 


0 


3+5 


1 


2 


0 


0 


0 


0 


1 


2 


0 


0 


4+5 


Q 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1+2+3 


1 


17 


4 


36 


5 


93 


4 


68 


0 


0 


1+2+4 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1+ 2+ 5 


Q 


0 


0 


0 


0 


0 


0 


0 


1 


5 


1+ 3+4 


1 


8 


4 


26 


2 


20 


5 


45 


1 


5 


1+ 3+ 5 


0 


0 


1 


6 


0 


0 


0 


0 


0 


0 


1+ 4+5 


1 


7 


0 


0 


0 


0 


0 


0 


0 


0 


2+3+4 


3 


15 


3 


24 


0 


0 


0 


0 


0 


0 


2+ 3+5 


2 


17 


0 


0 


0 


0 


0 


0 


0 


0 


2+4+5 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


3+4+5 


i 


5 


0 


0 


0 


0 


0 


0 


0 


0 


1+2+ 3+ 4 


0 


0 


5 


104 


3 


79 


2 


41 


4 


69 


1+2+ 3+ 5 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1+ 2+4+5 


1 


4 


0 


0 


0 


0 


1 


17 


0 


* 0 


1+ 3+4+ 5 


□ 


0 


0 


0 


0 


0 


0 


0 


0 


0 


2+ 3+4+ 5 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1+ 2+3+4+ 5 


1 


23 


5 


134 


3 


259 


3 


227 


5 


3 87 


Total 


289 


509 


87 


509 


22 


509 


46 


509 


19 


509 



1 - Decision makers 

2 - Plotters 

3 - Sensor operators 

4 - Continuous tasks 

5 - Discrete tasks 
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tim-tvc’v AisiA "LYSIS For the complexity index analysis, only 
COMPLEXITY INDEX ANALYSIS. ^t e _ory comroon areas accounted for sub- 

14-2+3+4 and 1+2+3+ P * /rrak i e Codes found In these two areas 

stantial numbers of tasks per ^ representative of 46. 7 

of commonality contain _-P coded levels in these intersections 

percent of the observed tasks. Jj£, ^i^ndlcates that the higher levels of 

rSmSexity Ind information processing are not common to all operator cate 
gories, 

^m^^h^wed^^a^^^n umber ^of^asks^l^^accminte^for^by^th^ee^c^es* found 

fntlm^+2-f3+4+5 °P era tnr oategor^ overlap^area^ Th^ese^^odes^represent^^^ 

¥& wSrie^s a oTrnce S rta g inty (Code Levels 5 and 6) were normally ^ 

cnc.NITIVE ELEMENT ANALYSIS. Analysis of cognitive elements above 
sh^,ed T that44. 6 percent of the task 

E^srSsK* ftSSjg&lSsS&Z- 

concept formation and generalization were in the cones comm 
1, 2 and 3, 

RESPONSE ELEMENT ANALYSIS. The analysis based “" the^ response ele- 
ments alone indicates that the majori y o _ _ * , g (see Table 5). 

™eTasL?nXded the full range of observed modalities and complexities. 

A PROFILES Th0 fr6QUency with which levels of various S GT -R 

category based on this analysis. 

TART F 6 MODAL S-C-R ELEMENT LEVELS FOR 
TABLE . opERATOR CATEGORIES WITHIN DEVICES 



— — p— 


Device 


Category 




1 


2 


3 


4 


5 


20A62 


15-232-22 


23-312-33 


13-312-33 


23-212-22 


23-332-23 


21A38 


22-312-22 


22-312-32 


22-312-22 


22-312-32 


22-312-33 


2F69B 


12-323-22 


12-312-34 


12-312-34 


w — — 


L- — — — 
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This table shows the most frequently occurring task code levels. In 
Devices 2QA62 and 2F69B the most common task code levels for Operator 
Category 1 personnel indicate heavy reliance on visual stimuli. The decision 
makers in Device 21A3S (Category 1 personnel), on the other hand, rely much 
more heavily on aural stimuli. The primary use of visual stimuli by decision 
makers in Devices 20A62 and 2F69B reflect, to some degree, the envirornnent 
within which these individuals function. For Device 20A62, the OOD pre- 
dominantly used the bird's eye CRT as his tactical display. For the TACCO 
in Device 2F69B, the major display was again a CRT. For the decision 
makers in Device 21A38 (CONN, FCC and Sonar Supervisor), however, no 
such central display was available. Thus, verbal communication was the 
predominant method of interacting. 

Table 6 also shows the high degree of similarity in the levels of task 
elements for the various operator categories. The preponderance of levels 2 
and 3 for all S-C-R elements suggests that the majority of tasks accomplished 
by various operators in the study devices are neither difficult nor complex. 
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SECTION IV 

DISCUSSION AND IMPLICATIONS 



The objective of this study effort was to determine how advanced technolo- 
gies SUC h as generalized or adaptive training could be beneficially applied to 
law tacticaLteam training. The team-task data were analyzed to determine 
whether sufficient commonality exists among tasks to warrant recommending 
a generalized approach to tactical team training, ( 

TASK DATA 

Tn this study the degree of task commonality varied as a function of the 
molecuSrlty oFthe analytis. At the most complete level of analysis, where 
“ach of the seven taxonomic elements had to match perfectly for commonality 
to exist little commonality was found. The comparison of complete task 
codes did reveal, however, that there was more commonality 
formed in devices containing weapon systems (Devices 21A3 8 and 2F88©) 
between either of these and the maneuvering trainer (Device 20A62), The 
apparent reason for this finding is the expanded role of the sensor operators 
as well as the presence of fire control equipment operators m the weapon sys- 
tem trainers. 

The frequency of occurrence of task element levels was tabulated by 
device and by operator category within device. These analyses, which ignore 
the sequential relationship between task 3-C-R elements, produced device and 
operator category profiles. Such information is useful for the design of any 
training system for it delimits each operator's task environment. Develop- 
ment of total and part-team trainers requires that appropriate task stimulus 
and response modalities be identified along with an estimate of their capability. 
In addition, the cognitions which mediate the task stimulus and response must 
be identified if they are to be effectively trained. 

The device profiles obtained In this study differed from one another 
primarily in terms of stimuli and responses. Whereas tasks performed m 
Devices 20A62 and 2F69B had predominantly visual stimuli, the task responses 
were mostly verbal for Device 20A62 but motor for Device 2F69B. The modal 
task code profile for Device 21A3 8 showed correspondence between both stimu 
lus and response elements (i. e. , a discrete aural input and a discrete verbal 
output). Thus, if a generalized training scheme is considered, it should 
allow for the adjustment of the Input and output task parameters to correspond 
to those relevant to the current training and presumably operational 



Contrary to the profiles for stimuli and responses, the modal task code 
profiles of all three study devices contain the same cognitive element levels 
(viz. -312-) corresponding to Perceptual Recognition, Data Analysis, and Rule 
Following. These elements describe the characteristics of a procedure 
following task. If the greatest commonality is that the devices all trained 
procedural skills, this suggests that generalized procedural- skill trainers 
might be feasible. However, before this can be recommended, three ques- 
tions must be answered. First, do the partial code analyses indicate a 
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similar commonality among operator categories? Second, Is the acquisition 
of procedural skills the desired terminal objective of the tactical team train- 
ing system? Third, what ir the status and conditions of use of existing 
trainers? To answer the first question, the partial code analyses will be 
reviewed. 

The Complexity Index analysis showed that task elements common to all 
five operator categories were of relatively low level, i. e, , tasks at the level • 
of explicit information exchanges, specific orders, or acknowledgements. 

Each of these is a highly specific event with minimal uncertainty. Tasks 
which embody high uncertainty, such as extracting a signal from a noisy back- 
ground, were found to be common only to the decision maker and sensor oper- 
ator categories. Thus, subsets of operator categories can be formed on the 
basis of the Complexity Index with decision makers and the sensor operators 
in one subset and the talkers, helmsmen, etc. (Category 4 and S personnel) 
in the other. Plotter (Category 2) personnel performed tasks which were 
most similar to operators in Categories i and 3, and are thus placed in the 
former subset. The nature of Operator Category 2 tasks, however, was 
heavily influenced by the large number of different tasks performed by the 
navigator at Device 2F69B, 

Further support for establishing subsets of operator categories is pro- 
vided by the analyses of the stimulus, cognition, and response elements. With 
the exception of environmental inputs to sensor operators, aural task stimuli 
were found to be quite specific in nature. These stimuli were common to all 
operator categories, indicating that verbal exchange was the primary medium 
of team interaction. Complex visual and indeterminate stimuli were common 
only in the decision maker and sensor operator categories. Thus, the objec- 
tives of and facility for training these personnel should include provision for 
complex patterns of stimuli, features typically associated with high fidelity of 
stimulation. 

The analysis of response elements indicated a great deal of commonality 
among operator categories, with a predominantly verbal modality. Unique 
responses were associated only with the motor requirements of the sensor 
operators’ and navigators' tasks. 

Cognitions common to all operators were fundamentally procedure- 
following. The task codes involving cognitions thought to be involved in tac- 
tical decision making, such as the development of a unique battle plan or 
innovative behavior, were few in number. However, when these codes were 
observed, they tended to be associated with the decision-maker category, 
either uniquely or in common with the sensor operators. The fact that the 
higher-level cognitions congregate in one subset of operator categories further 
supports a concept of differential training. 

Thus, the commonality observed in the device profiles is not completely 
supported by the partial operator category analyses. Based on the suggested 
division of operator categories, at least two types of training are indicated. 

The support personnel in Operator Categories 4 and 5 exhibited task behaviors 
which do not require a high-fidelity training device. The individual skills 
necessary for these tasks would benefit most from on-the-job training (OJT). 
Their preliminary team skills could also be trained aboard ship by means of 
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talk-through problem scripts, cross training, as well as OJT, Decision 
makers, sensor operators, and plotters, however, perform tasks which could 
benefit most from training facilities and techniques currently not available 
except at shore-based training devices. Thus, a part-task trainer for these 
personnel appears warranted. Such a trainer should emphasize the ambiguity 
of the operational environment to enable the acquisition of individual and basic 
team skills prior to entering the team training situation. 

The second question posed earlier is whether training teams to cope suc- 
cessfully with high-risk environments should culminate in the training of pro- 
cedures. The literature (cf. Alexander and Cooperband, 1965j Sidorsky and 
Houseman, 1966) says that the answer should be "no. " The emphasis of tac- 
tical team training should be upon generalizing from procedural skills to deal 
with novel and unexpected environments and contingencies. Yet, the emphasis 
of the training observed in this study never reached the level where teams 
were applying creative responses to ambiguous situations. 

The third question posed at the onset of this study was what are current- 
use practices at existing devices. The following subsection answers this 
question in light of the observations made during this study. 

CURRENT TRAINING 

The data and observations of this study indicate that Navy tactical team 
training is not achieving its required or potential effectiveness. It was found 
that intact teams seldom appeared for training. Instead of the established 
teams who normally operate together as a unit, the majority of "teams" 
trained in the study devices were simply groups of individuals formed into ad 
hoc teams at the training device. Specifically, Device 20A62 was used tc pro- 
vide practical experience for personnel attending Rules of the Road courses 
and also for those in Officer of the Deck (OOD) school. Rules of the Road 
classes were composed of trainees from or enroute to operational ships. 
Students from the OOD school were junior commissioned officers, many of 
whom were preparing for their first assignment at sea. Generally, then, 
personnel being trained at Device 20A62 were assigned there to learn basic 
skills rather than to develop the team coordination which assumes this knowl- 
edge as a prerequisite, 

A similar situation existed at Device 2F69B, The major user of this 
device was a Replacement Air Group (RAG) squadron. This squadron provided 
a 10- week course to prepare individual officer and enl isted personnel for 
operational billets. The secondary users of Device 2F69B were crews of 
operational aircraft. During the observations of this study. Device 2F69B 
was used either to qualify individual crew members or as an alternate site 
for required operational readiness inspections. 

The use being made of Device 21A38 presented an exception. This device 
was observed to be used exclusively by teams who normally functioned together 
on FBM submarines. These teams used Device 21A38 on a regular schedule 
as a part of their required training cycle. 

However, even when full teams did appear for training at a study devic e, 
team members often did not possess the prerequisite skills for the training of 
team tactics. The instruction provided in these trainers, rather, concentrated 
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on teaching individual trainees how to operate their various equipments and 
the basic procedures necessary to interact in the team setting. These indi- 
vidual procedural skills mus t be part of the trainee's repertoire, at least to 
some reasonable degree, before the goals of team training can be undertaken. 

An additional observation was that instructor personnel were normally 
not adequately prepared for their jobs. Each instructor observed was a 
subject-matter expert in his major area (e, g. , shiphandling, ASW, etc, ), 
Some instructors had taken courses at a special Navy school to prepare them 
for instructional duty. However, based on the training methods observed, it 
is concluded that these instructor's courses are not meeting the need of pro- 
viding training specialists. Specifically, instructors were generally unaware 
of the influence of their actions on trainee behavior. Little thought appeared 
to be given to how the manipulation of problem parameters or the information 
provided during critique sessions could Influence the effectiveness of a given 
training experience, 

A major deficiency in the current use of study devices was that the basic 
materials required for an effective training program were either not avail- 
able or not used. These materials should include well defined and validated: 

• Objectives of training 

• Criteria of performance 

• Measures of performance 

• T ests 

• Problem scripts 

• Data base for performance evaluation 

c Evaluation procedures 

The stated objectives and observed performance measures and criteria 
used at the devices in this study are hown in Tables 7 and 8, respectively. 

As may be seen from Table 7, the objectives were stated in the form of a 
course overview. In no case were specific end-of- training performance 
objectives enumerated. Furthermore, the criteria and measures of perfor- 
mance used in these devices changed as a function of what an instructor 
viewed as being most important. Observations made during this study also 
indicate that no formal performance testing is occurring at these devices. 
Without measuring trainee or team performance, it is not possible to assess 
the effectiveness of training provided. 

These findings illustrate the current use being made of study devices and 
highlight some of the problems existing in the Navy team training environment 
None of these findings, however, are unique to this study, as all have been 
reported by other researchers (cf. Schrenk, et al, 1969; Jeantheau, 1969), 
The point to be made is that such findings reflect problems which pervade the 
entire Navy training environment. It is suggested that to solve these prob- 
lems, the or'sntation and use of the existing Navy training system be re- 
examined with the objective of determining ways to increase system 
effectiveness. 
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TABLE 7. FORMAL OBJECTIVES FOR DEVICES STUDIED 



20A 82 -- Emergency Ship Handling Trainer 



"To oresent a comprehensive review of the Rules of the Road 
(nautical) and practical application of these rules m the Conning 
of a ship in various situations (Ordinary and Special Circum 
stances? which will aid Conning Officers in preventing collisions 
at sea" (Curriculum, COMTRAPAC COURSE K00- 607, Rev. 
Jan. 1 



21A38 — Submarine ASW Attack Teacher 

"To develop the proficiency of Submarine Battle Station and 
Watch Station Fire Combat Parties" (COMSUBPAC INST 
1500. 15C) 



2F69B -- P-3A Weapon System Trainer 

Specific course objective not available. 
General use objective available in: 



COMFAIRWINGSPAC INST 1500. 3 and 1500. 4 (Conf. Suppl. ) 
OOMNAVAIRPAC INST 3131, 10 and 1510 series ^ 
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TABLE 8, PROFICIENCY MEASURES AND CRITERIA 



Unit 


Device 
2 0A62 


Device 

21A38 


Device 

2F69B 


T lam 


Measures (per ship) 

Objective - signals 
(whistle) intership 
distance speed, course 

Subjective - hazards 
occurred, concurrence 
with 11 rules of good 
seamanship 11 

Criteria (per ship) 

Subjective refei ence to 
rules of road. NWP-3 8. 


Measures 

Objective - Hit/ Miss 
target 

Subjective - team 
coordination to solve 
FC problem 

Criteria 

Objective - computer 
determined hit/miss 

Subjective - doctrine 
experience 


Measures 

Objective - Hit (not often 
used) | 

Subjective - none 


Subteam 


No Subteams in 20A62 


Measures 

Objective - P/K solution 
for target range, 
bearing course and speed 
(fire control) 

Subjective - Successful 
tactical maneuver (OOP), 
evasive action (OOD), 
time to detection (sonar) 
plot accuracy and time 
(plot) 

Criteria 

Objective - computer 
F/K solution 

Subjective * doctrine 
experience 


No Subteams in 2F89B 


Op er at or 
1 


Measures 
None 
Criteria 
Same as team 


Measures 

Objective - none 

Subjective - equipment 
operation communication 

Criteria 

Objective - none 

Subjective - doctrine/ 
experience 


Measures 

Objective - target 
signature (JEZ) 

Subjective - operation 
of equipment (TACCO, 
NAV, JEZ, JULIE, 
Radar) -- 

-- Tactics (TACCO) 

-- procedure (all) 

Criteria 

Objective - none 

Subjective - instructor 
exp' 5 ce 
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in sum, the data of this 

training is no t the essence of those required in the tactical 

importantly training for different operator categories 

environment, further, p ^ Finally, the most immediate and largest 
appears to be more' k>ene . ^ \ rgLining WO uld not be realized through develop- 
improvement in tactical te " more efficient use of existing 

ment of a generaiized traimng sy ' l td advanced technologies should 

deV i C ^X P resuYtfd?sired tL balance of the report will discuss principles 
produce the re __ „*?**■*• significant promise or have been empirically 

demon^rated^^hicrea^se^raining 0 effectiveness as they relate to the study 
devices. 

pprhans the best documented finding relating to increasing trainee pro- 

^Cie^r Thfs deficiency was staff 

personnel, ^ecommendat^n^were 3 mad^for ^acticm^ that would serve to 
improve the existing situation, 

a+ no.Hf'P PDA 62 it was recommended that an additional trainee/ 
instr^r^ercomrnunication systen | 

the imrdwar e U c onffgurat ion at Device 20A 63 ‘--^"tnst^cfors nor- 
instructor / stottonl^Consequently , each 

^anTns^u^tfr^^red to --tnujcate with a apeci^ £*£***• A 

nuwber & of C iMtances l ^e^e observed ^^re^structors ^ould have sub^tan- 

tiluy increased the utility of a training problem by proven* con 

* directions to a single trainee unit. 



xax. essa = 

station at Device 20A62, an additional ' b rldge units during training. 

^y^so S do^ng, ll £mUons 0 cr^ti^al in the problem solution could be determined and 
tim critiques accurately directed toward these events. 

• . • f_ nrr . thp staff at Device 2QA62 subsequent to the 

data ^col l^cH o^i 'phase ^ StUd ^,formami ^ o^SSuve p^ 
r^fnL'^^fnt 1 ^ ' ^ 20 aJ.' sokere is no 
way to determine the extent of improvement. 

A suggestion made “V res P°"^ '*» e a kTbtcV^e^ 

dirkte^kward^sob^ng a° WerS 

required to critique * h f _ p ®^ situSton requiring frankness in identifying the 
instructor , wa ® A office”* “his situation was often handled by the 

instructor addressing such errors indirectly, thus avoiding direct 
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confrontation or by totally ignoring the error. Neither solution Is appro- 
priate or effective for providing the required performance feedback. It was 
suggested that such situations could be avoided by having either officer or 
civilian instructors lead critique sessions when officer trainees were involved. 

The suggestions made at these two trainers were for actions which could 
be taken independently by trainer personnel, involved little or no cost and did 
not require lengthy development before implementation. 

Perhaps the major problem faced by users of Navy team trainers is the 
lack of requisite training software. These include a specification of the 
objectives of training, performance criteria, evaluation tests, device use 
euldes etc. Until such materials are developed and used there is no objec- 
tive way to assess the adequacy of the training. Thus, a major miprovement 
in the effectiveness of all team trainers can be expected from the development 
and use of measurable behavioral objectives of training. Performance criteria 
and associated test materials must also be developed to permit the assessment 
of that training. 

Trainees were often not adequately prepared for team training. Our 
observations indicate that this situation could be substantially improved 
through the use of additional individual and/or part-team training. A variety 
of training media can be used both ashore and aboard ship for this purpose. 

In some cases media can be combined to produce a more efficient training 
program than any single medium alone might produce^ Such multi-media 
approaches have received considerable attention (cf, Briggs, Campeau, 

Game and May, 1 967 ) but cannot be realistically implemented without 
careful study of the individuals to be trained and the skills to be acquired. 
Further the sequencing of media has been found to be as important as their 
selection (Parker and Downs, 1961), As Rhode et al. (1970) have concluded: 

"Research and experience have failed to establish the sorts of 
definitive and meaningful relationships between media and general 
objectives that provide the basis "or any firm and final conclu- 
sions in this regard," (p, 297). 

Rhode et al, have compiled extensive information on the following 
instructional media: 



• Portable instructor aids 

• Television 

• Student response systems 

• Dial access information retrieval systems 

• Learning laboratories 

• Programmed texts 

• Teaching machines 

• Simulators 

e Computer-aided instruction 
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Rhode, et al. reviewed each of these media with respect to their 
instructional flexibility, support requirements, and relative costs. Such 
data can be used to define cost-effective approaches to the training of both 
individuals and teams in tactical skills. This, however, is beyond the scope 
and time limitations of the present program. Instead, possible applications 
of selected technology and media which offer significant promise to produce 
more effective tactical team training will be briefly discussed. 

A review of the literature (military and civilian studies and applications) 
indicates few techniques immediately applicable to team training. The 
majority of training technology is geared toward individual instruction. How- 
ever it is obvious that enhancing individual trainee capabilities prior to their 
entering the team training environment will do much to further the efficient 
use of the team environment. Although many of the technologies reviewed 
here pertain to individual training, emphasis will be devoted to those tech- 
niques that offer the greatest potential for practical implementation. 

One advanced training technology concept is Learner- Centered Instruc- 
tion (LCD- LCI is characterized by course content and training devices 
developed from the job’s behavioral requirements. State-of-the-art instruc- 
tional media and methods are selected to provide maximum practice of the 
relevant behaviors. With LCI, trainees work at their own pace, with the 
instructor no longer the focal point of training, 

V&lvfirdfi and Pieper (1970) developed and evaluated an LCI course for 
Air Force electronics maintenance skills. As part of their program, they; 

• Performed a task analysis and identified the learning objec- 
tives (training requirements) 

• Developed a job performance test to measure the trainee’s 
ability to perform the job behaviors 

• Developed a job- specific simulator for testing and teaching 
job behaviors 

• Designed a multimedia, job-specific course 

® Compared performance on the LCI course which taught the 

required job behaviors with a conventional course which dealt 
primarily with electronic principles of equipment operation 

Their evaluation indicated that significant performance and cost benefits 
were derived from application of the LCI program. The principal benefits 
occurred with high- aptitude trainees, both at the end of training and after five 
months in the field, 

LCI appears to be most appropriate for training individuals prior to their 
entering the team training environment. Those tasks for which the job 
behaviors, stimuli, and the responses are well identified are most amenable 
to this approach. Therefore, due to the procedural nature of jobs accom- 
plished by all team members observed in this study, significant benefit would 
probably result from application of this technique. Depending on where the 
training occurred (onboard ship or at a shorebased device), a variety of 
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media might be used. These include programmed books, part- and whole- 
task simulators, nonfunctional moekups, lectures, and teaching machines. 
Certain of the more promising of these materials and techniques will be dis- 
cussed with the intent of indicating how they could be applied. 

Television, as an instructional medium was considered by the Navy as 
early as 1951 (Rock, Duva, and Murray). Recently, the Navy's Personnel 
and Training Research Laboratory conducted a thorough survey of training- 
related uses of television (Hansing and Matlock, 1970). This survey leads us 
to consider a number of applications of this technology to the problem of ship- 
board and shorebased team training. 

The split- or multiple- screen technique procedure could be used to show 
team members their jobs in relation to other team members (cf. Krickel, 
1984). A trainee (or trainees at a given station) could observe how a given 
action serves as a stimulus for some other team member. Similarly, a 
team member could determine the basis for some input that he receives from 
another station. Such training provides the trainees with a better apprecia- 
tion for their role in the team and of the need for effective communication and 
interaction. For example, this technology appears to be especially appro- 
priate when used in conjunction with video tape to show how members of 
"good" versus "bad" teams interact. Videotaped television can be used for 
establishing a team awareness in team training situations where portions of 
the team are physically separated from one another (e. g. , Devices 21A38, 
14A2, etc, ), 

The videotape lecture technique generally allows for uniform polished 
presentations, economy of instructor time, and scheduling flexibility. Most 
studies have shown that retention is at least as good with this method as with 
conventional lecture methods, and it generally is well-received by students 
(Hansing and Matlock, 1970). TV classroom training has already begun at the 
Fleet ASW School, San Diego (Hansing and Matlock, 1970), 

The use of videotape also provides the capability to provide immediate 
feedback to a trainee after he has practiced some response or procedure. 

This is a unique advantage of this medium over motion pictures. The Air 
Force has recently applied this technique effectively for individual pilot 
training on the T-37 aircraft (Valverde, 1970). Videotapes permit compari- 
son of different stages in a trainee's progress and can be used in the shore- 
based facility to refresh instructor's memories about an individual's or 
team's performance. Another application of the split-screen technique dis- 
cussed earlier would be to display the trainee's (or team's performance) 
simultaneously with a "perfect" maneuver or task performance. 

A further use of closed- circuit television and videotape techniques is 
the monitoring of training from aboard ship. This would permit the training 
facility to be extended to include additional subteams. For example, a bridge 
unit of the 2 0A62 could be linked to the team's own Combat Information Center 
aboard ship. Also, key personnel such as the Commanding and Executive 
officers could participate in or observe and critique training without having 
to be present at the shore facility. 

Although the costs of such applications may be high, it appears that this 
technique has much to offer. 
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Many of the capabilities of the computer are highly applicable to team 
tactics training- Certainly, with the availability of digital computers in 
weapons systems trainers (e, g, , Device 2F87 and the 14A2 series), the 
possibilities are great for allowing the computer to perform many functions 
presently done by the instructor. 

Although individuals are ideally trained to operate as part of a smoothly 
operating team, their individual training seldom includes the acquisition of 
the interactive skills required in the team setting. One attractive approach 
to initiating interaction skills earlier in the training cycle is through the use 
of synthetic or simulated fellow team members. Here, the actions (inputs 
and responses) of other team members can be stored on magnetic tape and 
played to the trainee at the appropriate time in his operation (cf. Blake and 
Brehm, 1954). In this way the individual operator trainee can develop the 
required individual skills in a situation more closely paralleling that which 
will ultimately exist. Data from Klaus and Glaser (1970) indicate that such 
simulated team member training should be supplemented later by actual team 
interaction. 

Another function the computer can perform is to administer proficiency 
tests to trainees when they enter the team training environment. These tests 
can help determine which members are deficient in skill, and allow the com- 
puter to match a program of instruction to the trainee characteristics. A 
computer-based individual training system for decision makers or Category 1 
personnel (OOD, TACCO, Attack Officer) is described below. 

The system might be composed of a computer- driven cathode ray tube 
(CRT) and function keyboard. On the CRT could be displayed a symbolic 
view of an ASW situation. Appropriate math models would be used to portray 
ship's motions, sensor capabilities, and weapons effects. Various scenarios 
of different predetermined problem difficulty could then be programmed and 
displayed to the trainee, allowing him to practice procedures related to sensor 
and weapon deployment. The trainee could also make tactical decisions 
required in the ASW or other tactical environment, while of course risking no 
loss of men or material. In effect, this advanced training concept compresses 
6xp€ri6nc6 for an operator, yet offers many of the benefits of computer— aided 
instruction (e. g. , low student/ instructor ratio, immediate feedback, moti- 
vated trainee, provisions for branching), Siegel and Federman (1970) per- 
formed an analysis of the TACCO' s intellectual functions and derived a 
conceptual device embodying many of the above characteristics. Additionally, 
Honeywell has developed as a test vehicle such a computer-based system for 
the ASW officer. In sum, this concept seems especially appropriate for 
use in improving individual skills prior to a team session. Here, after a 
predetermined level of individual proficiency has been attained, the decision- 
making team member could enter the team context with its greater ambiguity 
and uncertainty. Application of this technique, whether aboard ship or at a 
shorebased installation, could, for example, substantially reduce the problems 
caused by deficient TACCO trainees at Device 2F69B. 

To avoid wasting the team training session because one or more team 
members come unprepared for team training, it may be advantageous to 
prompt individual operators within the team context. Prompting has been 
used effectively to train individuals in sonar discrimination (Annett and 
Peterson, 1967), visual identification (Wets z and McElroy, 1964), and 
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auditory detection (Annett and Clarkson, 1964), The techniques developed 
for the experimental studies cited above could provide the basis for prompting 
in the team setting. As an example, all fire control solutions for weapon 
delivery systems depend heavily on the accuracy of the sensor operator's 
tracking ability. If a sonar operator were in error beyond some predeter- 
mined amount, the computer controlling the trainer could be preprogrammed 
to automatically prompt that operator. Such prompting could appear in the 
form of a more easily tracked brightened target trace, the appearance of a 
reference dot at the appropriate tracking cursor position, etc. As a result, 
the deficient tracking operator would be coached into accurate behavior, and 
the team training exercise would not be degraded. This concept is applicable 
to all operator categories, not just decision makers and sensor operators. 



A capability of automatic monitoring of trainee proficiency and progress 
would allow the consideration of a technique known as adaptive training. In 
adaptive training, task or problem difficulty is automatically adjusted as 
function of trainee performance level. The requirements for an adaptive 
system are (Kelley and Wargo, 1968); 

© Valid and reliable performance measures 

• At least one system, task, or environmental variable that 
affects task difficulty 

• Adaptive logic which automatically adjusts task difficulty on 
the basis of the relation of measured performance to present 
criterion of performance 

To date, the greatest amount of research on adaptive techniques has been 
in the area of continuous perceptual-motor skills (cf. Kelley and Wargo, 

1968), Research on detection -- identification tasks (e. g. , Mirabella and 
Lamb, 1966; Swets, Millman, Fletcher, and Green, 1962) have yielded 
somewhat equivocal results. 



Many problems must be solved before adaptive training can be effectively 
implemented in the team, or any training situation (Caro, 1970; Leonard, 

Doe, and Hofer, 1970). Particular problems involved in using adaptive tech- 
niques for the team tactics situation are; (1) establishing valid and reliable 
performance measures and (2) combining separate aspects of performance 
measurement into a single continuum for adaptation. These problems continue 
to receive much attention from workers in the field^. 



The concept of adaptive training could be implemented by a competent 
instructor. To do this, more adequate provisions must be made for monitoring 
and evaluating team performance. These provisions would require expansion 
of, the computer capacity at the devices of the present study. Further, 
instructor personnel must receive appropriate training in adaptive techniques. 
Using the computer to collect and analyze performance data appears to be the 
only reasonable means of attaining truly standardized measurement in the 



■^Adaptive Training -- Discussion transcript conducted on 2 7 and 28 April 
1970 at the University of Illinois Institute of Aviation. 
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team setting. Furthermore, two current problems can be solved through 
automation. First, automatically collected performance measures are less 
likely to be challenged for accuracy, thus removing the current need for 
instructors to defend their criticisms of team functioning. Second, instructors 
are freed from the task of manually recording performance data and can more 
fully use their system knowledge to assist the team in reaching proficiency. 
Some attempts at incorporating such real-time instructor aids have been 
made with the Army’s Synthetic Flight Training Simulator (Prophet, 1070). 

Certainly, the data base which could be collected with an automatic 
performance- monitoring capability could also have another important applies 
tion, namely, to allow for evaluation and evolution of the training system. 

Any* effective training system must remain open to refinement through evalua- 
tive feedback and validation. As trainees become more proficient, the 
training curriculum must be adjusted. Allowing the computer to record 
trainees' performance and to store and compare team data longitudinally 
would permit long-term changes in overall performance to be detected and 
acted upon. In addition, individual training exercises can be analyzed for 
their effectiveness in training requisite behaviors (i, e. , item or scenario 
analysis). 

This discussion has served to point out that the development of a 
generalized team-tactics training system is not appropriate at this time. 
Rather, significant improvements to the existing situation may be made by 
the application of advanced technologies. The next subsection offers specific 
suggestions for achieving these improvements. 

IMPLICATIONS 

A conceptual view of the process from individual to tactical team training 
is presented. At the level of individual operator categories there is much 
promise for general- skill trainers. The tasks required of sensor operators 
(pattern recognition, procedural following), and vehicle control operators 
(continuous or discrete tracking, procedure following) are amenable to training 
via computer-based, part-task trainers. Training the decision maker for 
tactical command involves a number of other skills (e. g, , individual pro- 
ficiency in situation recognition, knowledge of the procedures required to 
implement situation solutions, knowledge of his platform's capabilities and 
limitations, knowledge of enemy characteristics, etc. ). Thus, the training 
of this individual skill involves task-oriented problem solving. 

The next level of training would involve an assembled team. The 
emphasis would be on interaction and coordination and the development of a 
sense of team awareness. The training could occur either on the job if given 
the appropriate software, or at the training device. The objectives at this 
stage would be to learn what information should be transmitted to other team 
members and how it should be transmitted. Something as simple as a special 
earphone through which the instructor could prompt selected individuals so 
that the information flow could be maintained could be used as an aid in 
meeting this objective. The decision maker should be trained to exercise 
leadership control of the communication networks. From individual training 
both the decision maker and the sensor operator should have acquired know- 
ledge of the basic stimulus patterns they will be exposed to and their asso- 
ciated procedural responses. Now they would acquire the ability to extract 
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such stimulus information from a more noisy but still relatively unam- 
biguous background and transmit it appropriately. 

Once the individual skills are mastered, and a sense of coordination or 
team awareness acquired, training can be concerned with the essence of 
tactics viz, dealing with uncertain, ambiguous, or emergency situations. 
This involves a whole team in the process of modifying and generalizing 
from established procedures to deal with an intelligent opponent or environ- 
mental hazard. When one thinks of training in this form, concepts such as 
the linking of a submarine and a surface and/or air ASW device are appro- 
priate suggestions. This would require, however, a fine degree of exercise 
control if this situation is to be in fact training rather than free play. 
Training tactics involves uniqueness. While general processes may be 
involved at an individual level, the team tactical setting requires new instruc- 
tional techniques and specific hardware capabilities but not necessarily new 

devices. 

Thus the degree of device and operator-task commonality cannot be 
considered independent of the presently available equipment or conditions of 
training It is concluded that currently available training hardware is more 
than adequate to train teams effectively, but that these devices are not being 
used as effectively as they might. Thus, rather than suggesting that the Navy 
embark on a costly developmental program for a generalized team tactics 
training system, it is recommended that better use be made of available 
facilities, 

effective training system reflects a planned and controlled learning 
environment. The planning of such an environment requires several steps- 

• Task and function analysis 

• Training requirements analysis 

• Training program development 

• Training device design 

• Training program use and evaluation 

• Training program revision 

A number of approaches have been developed and are available to 
accomplish these program elements. Each of the steps outlined above are 
necessary for the development of a new training system or the modification 
of an existing one. Briefly, the goal of each of these developmental steps is: 

• Task and Function An alysis -- Identify and reduce operator 
tasks, bh a mission- oriented basis, into segments which can 
be analyzed to determine training requirements. 

• Training Requirements Anal ysis -- Identify criterion tasks 
which should be incorporated into a training program to achieve 
desired terminal levels of performance. 
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• Training Program Development -- Define performance objec- 
tives; select subject matter and sequencing; define length and 
frequency of training; identify training methods; define instru- 
ments. 

• Training D evice Specifications and Design Development - - This 
step assumes that a new training device system will be 
developed to fill an existing need. Specify device characteristics 
and translate into hardware. 

e Training Program and Evaluation — Development of use proce- 
dures, guidelines, programs, evaluative instruments, etc. , to 
support training system. 

• Training Program Revision -- Based on a change in system 
objectives, availability of new technology, or indications of 
ineffective training from performance data base, revise 
training program to maintain or increase effectiveness. 

These steps, then, reflect a recommended philosophy which is applicable 
for the development or revision of any training program. 
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SECTION V 
RECOMMENDATIONS 



The data and observations of this study combined with the findings of 
previous team and generalized training research, lead to the following recom 
mendations, 

1. The Navy should concentrate on improving the effectiveness 
of existing devices rather than launching a development pro- 
gram for generalized training systems. Significant improve- 
ments could result from; 

a. The establishment of formal, quantifiable behavior 
objectives of training. This would provide a frame- 
work for the selection of training media, the 
sequencing of instruction, and the evaluation of 
trainee progress. 

b. The development of performance criteria. This 
provides the yardstick against which training effec- 
tiveness can be assessed. 

c. The development of appropriate test materials and 
use guides. Diagnostic and achievement tests would 
assist instructors to assess trainee proficiency upon 
entering the team environment and to determine an 
appropriate set of problem exercises. The use guide 
is a definite requirement. The utility of the sophisti- 
cated training systems that have been developed is 
limited by the ability of instructors to use them., 

d. More adequate instructor training. The initial train- 
ing of instructors should place greater emphasis on 
his role and impact upon trainee performance. 

Additionally, periodic critique and refresher training 
of instructors is warranted, 

e» Navy -wide evaluation procedures to assess the effec- 
tiveness of the materials, techniques, and utilization 
of team training devices. This would permit incorpor- 
ation of new training techniques, identification of new 
training requirements, and an earlier identification of 
training problems and potential problems. 

2. Specific recommendations for training device developme.it and 
use include the following; 

a. Teams should be required to appear at the training 

device as a unit. Thus, training could progress more 
rapidly toward the goal of improving team tactical 
effectiveness and the skills acquired in the trainer 
would transfer more readily to the operational unit. 
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Provision for automated monitoring, evaluation, 
and selective prompting and cueing of operators 
on an individual basis, within the team training 
context. This feature is oriented toward making 
the team environment more productive and reduces 
the detrimental effects upon team training of 
deficient individuals. 

c. Computer -managed instruction for decision makers, 
sensor operators and selected plotters. These 
could be general-purpose procedures trainers, with 
the provision for variable stimulus and response 
modalities and complexities. Further, these con- 
soles could be used for both individual skill training 
and combined for subteam training. 

d. Category 4 and 5 personnel should receive individual 
training on the job. It is inefficient to use the team 
training environment to teach steering and basic 
plotting skills. 

e. Total team procedures training should be conducted 
on the job. These behaviors do not require high 
fidelity of simulation and, thus, represent gn 
inefficient use of the team training device. 

f. Team tactical training devices should be oriented 
toward the training of the innovative behaviors 
relevant for dealing with emergent situations. This 
requires that trainees possess a prior thorough under 
standing of basic operating procedures. 

Training devices should be combined in terms of 
relevant adversaries, such as a submarine and 
destroyer combination. This training environment 
could be a highly effective way of simulating the 
operational tactical environment. 




has raised a number of questions concerning team tactics 



techniques for increasing training effectiveness. 




data from the operational setting. 
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A qpcond theoretical question is whether decision making is a generaliz- 
flhle i7 suggested by Sidorsky and Houseman (1966) and Kanarick 
M 9 l 9 ) The?e is a paucity of data on this question, yet it too underlies the 
program objectives^ What has hampered research m this area is the continu- 
ing need for an adequate criterion of decision-making performance. If 
decision making is a gener alls able skill, then optimal ways to tram the skill 
tin |nd should be determined. One way to compress tactical experience for 
can ana suggested earlier, vis, individual decision -making 

frai^iers "incorporating mathematical models of the operational situation com- 

generalized ^etrucUon hi learning to diagnose a tactical situation and to 
select an appropriate set of actions must oe empirically determined. 

Much has been said in this report concerning the current role of instruc- 
tors. The question remains, however, of the best use to which sucti highly 
Qualified subject-matter experts can be put in the instructional process. 
Should thly be required to initiate prompts and formulate performance 
critiques? Should they be on station with trainees or at their own cons e 
during the conduct of training exercises? How can they be better prepared 
"or ^ thlir duties as instructors? Although some suggestions for improvements 
have been made, questions remain unanswered as to the appropriate use of 
these personnel together with an indication of the man-hardware tradeo s 
which must be made in designing the automatic versus manual features for 

future devices. 

One training technique discusses as a possible solution to the problem 
of deficient team members was that of selective individual prompting. 
Quoqtions remain however, as to the specificity, schedule, and nature of 
such prompts Similarly, it should be determined what effects prompts have 
on°iron-promF>ted team Members and the Influence of ^cloue promptmg ^ 
schedules on transfer performance. Briggs and Jonnston <19 6 ^oi ^luded 
that specific feedback is detrimental early in training. Is the same true tor 

prompting? 

Simulating team members was suggested as one way to train certain 
interactive and communication skills, yet information is lacking on imple- 
mentation. The hardware and software requirements must be determined 
Ts a necessary first step in evaluating the cost-effectiveness of such an 
approach. 

The utility of adaptive training depends greatly on the measurement of 
Dprformance It has already been noted that adequate measure of team per 
formal e ar^ lacking Furthermore, the efficacy of adaptive training must 
be assessed for the highly proceduralized tasks observed in this stucy. 

Finally, multi-media instructional techniques are thought to be highly 
effective. Which media should be selected for the training of which jobs- 
Further, what should be their sequencing and relative durations. 

This set of questions is not meant to be exhaustive. Instead, it high- 
lights those areas where v/ell-conducted research could lead to the develop- 
ment (more effective utilization) of advanced technologies for future applica 
tion to team tactics training. 
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APPENDIX A 

CANDIDATE STUDY DEVICES (LISTED IN ORDER 
OF REPRESENTATIVENESS) 



Surface 

14A2 

20A62 

20A61 

14A6 

16B13 

1BZ2 

14A1 



Surface Ship ASW Attack Trainer 
Emergency Shiphandling Trainer 
Maneuvering Tactics Trainer 
ASW Coordinated Tactics Trainer 
Amphibious Operations Trainer 
Maneuvering Tactics Trainer 
Action Speed Tactical Trainer 



Subsurface 

21A38 

21B20 

21C4 

Air 

2F87 

2F69B 

2F66A 

2F84 

SFTS 

Other 

1 5F6 



Submarine ASW Attack Teacher 

Advanced Submerged Submarine Casualty Control Trainer 
Basic Submarine Diving Trainer 



P3C Weapon System Trainer 

P3A Weapon System Trainer 

S-2E Weapon System Trainer 

A7A Weapon System Mission Trainer 

Synthetic Flight Training System 

CIC Tactics Trainer 
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APPENDIX B 

TEAM MEMBERS BY POSITION NAME 



Device 

20A62 



Officer of the Deck 
Horizontal Plotter 



Radar Operator 



Helmsmen 
Lee Helmsmen 



Device 

21A38 

Conning Officer 

Fire Control Coordinator 

Sonar Supervisor 

Plot Coordinator 

Time -Bearing Plotter 

Strip Plotter 

Lynch Plotter 

BQR-7 Operator 

BQR-2 Operator 



Device 

2F69B 

Tactical Coordinator 
Navigator 
Jezebel Operator 
Julie Operator 
Radar Operator 



Attack Director 
Operator 

Analyzer Operator 

Weapon Control Console 
Operator 

Expanded Time-Bearing 
Plotter 

Time Range Plotter 
Time-Bearing Talker 
Time-Bearing Recorder 
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APPEND DC C 



DESCRIPTION OF METHOD 



LIAISON 

Initial contact with the staff of each device studied was made by the 
local NTDC representative. These NTDG personnel also furnished infor- 
niation concerning the organizational structure of the training unit being 
visited and the appropriate military command personnel to be contacted in 
arranging for data collection visits, 

TRAINER STAFF BRIEFINGS 

At each trainer site, members of the HST (Honeywell Study Team) 
provided a comprehensive briefing to the training unit command, trainer 
staff personnel, and NTDC representative. These briefings consisted of 
a description of the study rationale, objectives, methods, and anticipated 
results. From information obtained by HST. members following these 
briefings, existing trainer use schedules were established, and the specific 
times that study personnel would be in the trainer area were defined. 

TRAINER DEVICE FAMILIARIZATION 

Members of the HST familiarized themselves with the three devices 
studied through instruction in and/or operation of the hardware. These 
informal device familiarization sessions were conducted by instructor and 
maintenance personnel during periods when the devices were available for 
such activity. The familiarization process served two purposes. It 
allowed study team members to develop rapport with instructor and main- 
tenance personnel as well as providing them with the required information. 

TRAINING DEVICE DOCUMENT REVIEW 

This activity consisted of a review of training curricula, historical 
data, and training device descriptions. Lesson plans and training schedules 
were analyzed to determine the extent to which training objectives were 
specified in behavioral terms. Historical data was reviewed to determine 
the extent to which performance records were obtained and/or maintained. 
Finally, information pertaining to the theory of operation and capabilities 
of each device was collected for future reference. 

STRUCTURED INTERVIEW 

An interview guide was specifically designed for use in collecting non- 
task data. Using this sheet as a framework, interviews were held with 
trainer personnel who were selected for their knowledge in areas of special 
concern. For example, device staff and instructor personnel were ques- 
tioned regarding training objectives, evaluation, training methods, etc. 
Information concerning trainer equipment characteristics was generally 
obtained from the Training Devicemen (TD's) who were in charge of trainer 
maintenance. These structured interviews were quite detailed and generally 
required portions of two to three days to complete. 

O 
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APPENDIX D 
NUMERICAL TAXONOMY 



ELEMENT: STIMULUS 

CATEGORY I. MODALITY 



Level 


Code 


Description 


« 

None 


0 


Not used 


Visual 


1 


Stimulus perceived visually ~ seen. 


Aural 


2 


Stimulus perceived aurally - heard. 


Touch 


3 


Stimulus perceived tactually - felt. 


Combination 


4 


Stimulus perceived with more than a single 
modality -- stimulus may have visual, 
aural and tactual components. 


Other 


5 


No external stimulus present - - stimulus 
is internal to the man; e. g, , passage of 
time, uncertainty, vestibular, etc. 
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ELEMENT: STIMULUS 

CATEGORY II. INFORMATION UNCERTAINTY 



Level 


Code 


Description 


Noise 


0 


Only noise present as stimulus. 


Simple, one- 
bit, no uncer- 
tainty 


1 


An "on-off" type stimulus providing one bit 
of information with no uncertainty. 
Examples; Light is on or off 
Bell is on or off 
One word commands; "Now", 
"execute", "fire" 


Sample, single- 

parameter, 

discrete 


2 


Stimulus gives two or more bits of informa- 
tion from a small finite number of steps con- 
cerning one parameter with little uncertainty. 
Examples; Digital displays 

Discrete displays 

Verbal order for ship's course 


Simple, multi- 
parameter, 
discrete 


3 


Stimulus provides two or more bits of infor- 
mation concerning each of two or more dis- 
crete parameters with title uncertainty. 
Example; Verbal order for course and RPM 
sett ing 


Complex, multi- 
parameter, 
discrete and 
continuous 


4 


Stimulus provides two or more bits of infor- 
mation concerning each of two or more dis- 
crete or continuous parameters with moder- 
ate uncertainty. 

Examples; Moving, dynamic indicators 
Fuel consumption/ rate 
Speedometer - speed/ rate 
Altimeter 

Nonstandard verbal command 
Single aspect of a CRT display 


Complex, multi- 
parameter, 
continuous 


5 


Stimulus provides two or more bits of Infor- 
mation content about more than two dynamic 
parameters reflecting continuous steps from 
a very large finite number with unpredictable 
and moderate uncertainty. 

Examples; Ship's motion and transfer tables 
Multiparameter CRT or hardware 






displays (sonar, radar, OOD, 
TACCO) 

Aural sonar signals 
Discoursive verbal communication 
between two or more persons 
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ELEMENT: STIMULUS 

CATEGORY II, INFORMATION UNCERTAINTY 



Level 


Code 


Description 


Complex, multi- 
parameter 


6 


Highly complex, multiparameter stimulus 
which provides more than two bits of informa- 
tion and may contain high uncertainty due to 
masking, incompleteness, intermittent recep- 
tion, or not being displayed, A composite of 
discrete and dynamic information from an 
infinite number of possibilities with poten- 
tially high degrees of information when 
properly organized. 

Example: Tactical situation, "big picture' 1 . 



ELEMENT: COGNITION 

CATEGORY I. PERCEPTUAL PROCESSING 



Level 


Code 


Description 


Unidentified 


0 




Detection 


1 


Monitoring, attention, vigilance and detection 
of the stimulus against background noise. 
Stimulus presence is sufficient to initiate a 
response. Stimulus must be simple and 
readily perceived. 


Discrimination 


2 


Requires the simultaneous or sequential 
comparison of two or more detected stimuli 
in order to determine that they are the same 
or different: e. g. # differentiation, distinc- 
tion, compare and separate. 


Recognition 


3 


Stimuli require familiarity or unidentified 
past experience fo - " perception. 


Identification 


2 


Requires a naming or labeling activity, but 
the name of the stimulus is relatively unim- 
portant for task accomplishment. Name need 
not be specifically stated; e, g, , specific 
orders for maneuvering in a fog situation. 


Classification 


5 

i 


Requires a specific name which isolates the 
stimulus as a member of a specific category 
of events. Specifying the name is critical to 
task success. Assignment of the name may 
involve judgment. 

Examples: Sonar classification — submarine 

" Extremis" 

"Datum" 

- 



O 

ERIC 
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ELEMENT: COGNITION 



Level 


Code 


Description 


None 


0 


No processing of stimulus informat ion- 
conditioned reflex* 


Data Analysts 


1 


Cognitive activities of filtering, reducing, 
analyzing and cross comparing perceived 
stimulus data resulting in a cognitive listing 
of processed data with no organization. 
Example: Placing a cursor on the contact 

which is marked by a noisy 
background. 


Problem 

diagnosis 


2 


Requires identification of a problem through 
comparison of actual and desired state of 
affairs, weighting and enumeration of alter- 
nate states of affaiTS. 


Concept 

formation 


3 


Organization of the information produced as 
an output of problem diagnosis (2) resulting 
in the formation of a specific plan, idea or 
thought solution. 


Innovation - 
creation 


4 


Involves data analysis (2), concept formation 
(3) and the production of new information 
through generalizing from existing daca. 
Example: Development of a unique battle 

strategy , 

Development of unique plotting 
techniques from existing rules of 
geometry 
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ELEMENT: COGNITION 

CATEGORY III. ACTION SELECTION 



Level 


Code 


Description 


No action 


0 


Selects no overt action. 


Seeks information 


i 


Decides to actively seek additional informa- 
tion. 


Follow specific 
rule 


2 


Decides to follow a specific existing rule or 
procedure in making a response. 


Follow general 
principle 


3 


Decides to follow a generalized rule which 
may be based on existing guidelines for 
action; involves using common sense in t 
selecting a response. 



ELEMENT* RESPONSE 
CATEGORY I. MODALITY 



Level 


Code | 


Description 


Visual orienting 


1 


Looks at, using only head- eye movement. 


V erbal 


2 


Spoken or verbal sounds. 


Motor 


3 


All motor actions including eye-hand 
coordination. 


Combination 


4 


The chaining or combination of various 
response levels. 


Other 


5 


Used for responses which do not fit in other 
levels. | 
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ELEMENT: 



Level 



T 



j Code j 



Description 



Simple, discrete 



Controlled, 

single- 



discrete 



simple on-off type response requiring little 
or no skill beyond knowing when to respond. 
Examples: Button push 

One-bit verbal response 
Switch flip 

Requires a controlled, discrete act on one 
parameter such as giving a verbal RPM order 
This complexity is used when the response 
requires little skill beyond differentiating 
the relevant response from other similar 
response alternatives. 

Verbal RPM order 
Select single position of multi- 
position switch 
T.ook ud information in a book 



Controlled, 
multiparameter , 
discrete 



Similar to Level 2 but requires a controlled, 
discrete act on two or more parameters. 
Examples: Giving a verbal order and setting 

a switch 



Complex, skilled, 
continuous 



Requires sensory-muscle coordination. 
Example: Tracking 

Aiming 
Plotting 



Compound, 

multiparameter, 

continuous 



High skill, 
fine control 



Requires a long chain of discrete steps or 
a single continuous response. 

Examples: Procedure following 

Unstructured verbal discourse 

Requires very high skill levels resulting 
only from extensive practice. 

Example: Piloting an airplane. 
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